CompILE: Compositional Imitation Learning and Execution

We introduce Compositional Imitation Learning and Execution (CompILE): a framework for learning reusable, variable-length segments of hierarchically-structured behavior from demonstration data. CompILE uses a novel unsupervised, fully-differentiable sequence segmentation module to learn latent encodings of sequential data that can be re-composed and executed to perform new tasks. Once trained, our model generalizes to sequences of longer length and from environment instances not seen during training. We evaluate CompILE in a challenging 2D multi-task environment and a continuous control task, and show that it can find correct task boundaries and event encodings in an unsupervised manner. Latent codes and associated behavior policies discovered by CompILE can be used by a hierarchical agent, where the high-level policy selects actions in the latent code space, and the low-level, task-specific policies are simply the learned decoders. We found that our CompILE-based agent could learn given only sparse rewards, where agents without task-specific policies struggle.Comment: ICML (2019

A multi-agent reinforcement learning model of common-pool resource appropriation

Humanity faces numerous problems of common-pool resource appropriation. This class of multi-agent social dilemma includes the problems of ensuring sustainable use of fresh water, common fisheries, grazing pastures, and irrigation systems. Abstract models of common-pool resource appropriation based on non-cooperative game theory predict that self-interested agents will generally fail to find socially positive equilibria---a phenomenon called the tragedy of the commons. However, in reality, human societies are sometimes able to discover and implement stable cooperative solutions. Decades of behavioral game theory research have sought to uncover aspects of human behavior that make this possible. Most of that work was based on laboratory experiments where participants only make a single choice: how much to appropriate. Recognizing the importance of spatial and temporal resource dynamics, a recent trend has been toward experiments in more complex real-time video game-like environments. However, standard methods of non-cooperative game theory can no longer be used to generate predictions for this case. Here we show that deep reinforcement learning can be used instead. To that end, we study the emergent behavior of groups of independently learning agents in a partially observed Markov game modeling common-pool resource appropriation. Our experiments highlight the importance of trial-and-error learning in common-pool resource appropriation and shed light on the relationship between exclusion, sustainability, and inequality

De prospect a aluno: fatores influenciadores da escolha de uma instituição de ensino superior

As a consequence of the flexibilization of rules for launching undergraduate programs and schools in Brazil, occurred in 1995, an expressive increase in the supply of courses from private institutions was observed in the last 15 years. During such period, even though the number of Brazilian freshmen has more than tripled, a gradate increase in the difference between the supply of higher education and its demand is verified. This scenario aggravates when the fact that evasion indexes are increasing is taken into account. Thus, in face of pressures caused by the decrease in the percentages of occupancy rates, it is relevant and necessary for superior education institutions to identify marketing strategies that fit their needs to maintain and expand their activities. In such a context, the objectives of this quantitative study based on a sample of 226 high school seniors from Piracicaba (SP) were to identify the socio-demographic profiles of prospects in the region and the attributes that drive their choices for a higher education institution. The profile identified as having greater intention to join higher education programs is comprised by students who attend the private high school system, have around 17 years old, are white male and do not work. Still based on the data collected, it is suggested that effective university offerings and communications are those that fulfill the prospects’ expectations regarding campus localization, financial viability of the course, social life, infra-structure, technology, campus attributes, and institutional reputation, besides quality. Key words: marketing in education, higher education, consumer behavior, evaluation of choice alternatives.Como consequência da flexibilização das regras para a abertura de cursos e instituições de ensino superior no Brasil, encetadas em 1995, observou-se, nos últimos quinze anos, um expressivo aumento no número de vagas oferecidas por instituições privadas de ensino superior. Neste período, ainda que o número de alunos ingressantes no ensino superior brasileiro tenha mais que triplicado, verifica-se o aumento gradativo no hiato existente entre a oferta de vagas no ensino superior e sua respectiva demanda. Tal cenário se agrava, ao considerar-se que os índices de evasão de alunos vêm aumentando. Assim, frente às pressões de diminuição no percentual de preenchimento de vagas, faz-se relevante e necessário às instituições privadas de ensino superior que se identifiquem estratégias mercadológicas adequadas às necessidades de manutenção ou expansão de sua atividade educacional. Nesse contexto, mediante estudo quantitativo e com base em amostra de 226 formandos do ensino médio na cidade de Piracicaba (SP), o objetivo deste estudo foi identificar os perfis sociodemográficos dos potenciais estudantes universitários da região, bem como os atributos em torno dos quais a escolha por uma instituição de ensino superior se organiza. Constatou-se que o perfil de alunos com intenção de ingressar em cursos superiores é composto por alunos da rede particular do ensino médio, com idade em torno de 17 anos, do sexo masculino, brancos e que não trabalham. A partir dos dados coletados, sugere-se que as ofertas e comunicações universitárias efetivas devem ser aquelas que contemplem, além da qualidade do ensino, as expectativas de seus prospects em termos de localização, viabilidade financeira, vida social, infraestrutura, tecnologia, atributos dos campi e reputação da instituição. Palavras-chave: marketing educacional, ensino superior, comportamento do consumidor, avaliação de alternativas